DOCUMENT RESUME 



ED 260 758 JC 8 50 470 

AUTHOR Cohen, Arthur M. 

TITLE what Do Our Students Know? 

PUB DATE [85] 

NOTE 15p. 

PUB TYPE Reports - Research/Technical (143) — Viewpoints 

(120) 

EDRS PRICE MF01/PC01 Plus Postage. 

DESCRIPTORS Community Colleges,* Educational Assessment; *General 

Education; *Knowledge Level; Learning; *Liberal Arts; 
♦Outcomes of Education; Two Year Colleges; *Two* Year 
College Students 

IDENTIFIERS *General Academic Assessment 

ABSTRACT 

Recent research studies have focused on various wavs 
of addressing the question of how much students learn in community' 
colleges, focusing on factors such as transfer rates, job placement 
rates, and alumni satisfaction. Few studies, however, have attempted 
to directly confront the question of student learning, and few 
measures and tests are- adequate to assess what knowledge is being 
gained by student cohorts. A study conducted by the Center for the 
Study of Community Colleges (CSCC) attempted to demonstrate that a 
measure of what community college students know can be obtained. The 
Center developed the General Academic Assessment (GAA) instrument to 
measure student knowledge in general education and the liberal arts. 
The GAA was administered to 8,026 students in 23 two-year colleges, 
and additional information was obtained on students' age, 
aspirations, reasons for attending college, number of courses taken 
in disciplinary areas, total units completed, and self-assessment of 
knowledge. Study findings included the following: (1) student 
knowledge was related both to age and number of courses completed in 
a particular area; (2) age relationships showed up primarily in 
English usage, social sciences, and humanities, with older students 
having higher scores; (3) age-related differences did not occur in 
mathematics or science; (4) a direct relationship existed between the 
number of units completed and GAA scores; (5) highest scores were 
made by students attending college for personal interest; and (6) 
students* self-assessment of their knowledge showed high correlations 
with their GAA scores. The study confirmed the usefulness of the GAA. 
(lal) 
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Abstract 

As away of demonstrating that a measure of what community 
college students know can be gained, the Center for the Study of 
Community Colleges built and administered a General Academic 
Assessment in association with faculty and administrators in 
Chicago, Miami, Los Angeles, and St. Louis. Results of the test 
given to 8,026 students in 23 colleges in those districts were 
arrayed according to students 1 scores on subtests in humanities, 
science, social science, mathematics, and English usage, and 
related to the students' age, aspirations, reasons for attending 
college, number of courses taken in each area, total units 
completed, and self assessment. 



WHAT DO OUR STUDENTS KNOW? 

Arthur M. Cohen 

Opportunity with excellence, the motto of the American 
Association of Community and Junior Colleges, says it all. We 
want the community colleges to be accessible to anyone who can 
profit from attendance and we want them to be excellent in their 
many endeavors. Access suggests maintaining open admissions not 
limited by financial and scholastic barriers and excellence is 
helping students achieve their goals in programs suited to the 
local constituency. Excellence is further defined as "value 
added," the learning attained by the student while attending the 
college. 

This latter portion of the definition of excellence also 
receives attention periodically as a measure of institutional 
quality. For despite all the subordinate roles for community 
colleges, as educational institutions, they are of necessity 

judged by the value they add to [their students' intellectual 

i 

capabilities and moral qualities A school must teach, or what's 
it for? Students must learn, or what's it done?* While access is 
essential as a prime component in community college mission, 
students must eventually display some element 01 learning or goal 
attainment lest the access be a hollow victory* 

Easily said, less easily measured; how much do students 
learn in community colleges? The question is usually answered 
obliquely. Many colleges and some state systems maintain esti- 
mates of the success enjoyed by their matriculants who transfer 
to senior institutions; Radcliffe (1984), Illinois Community 



College Board (1984), Fernandez (1984), Doherty and Vaughan 
(1984), and Young (1982), are recent examples of such studies. 
Program planners in most colleges also have an idea of the number 
of students obtaining jobs in the field for which they were 
trained; see for examp le, Lee (1984), Scott (1984), New Hampshire 
State Department of Education (1984), and Lucas (1984). And some 
colleges survey their alumni periodically, asking if they were 
pleased with the experiences provided them by the college. Such 
studies are reported by McConochie (1983), Nespoli and Radcliffe 
(1983), Staatse (1983), and McMaster (1984). 

Curiously, the question of student learning seems the one 
measure of value added that is 1 east likely to be "answered* 
Comparing the grades earned by community college transfers with 
those made by students who began at the universities is of 
marginal utility because relatively few students transfer. 
Measures of job-gaining are better, even though most of the 
reports reveal serious methological flaws* Less useful are the 
studies in, which former students are asked if they felt they had 
learned. The answer to that question is too often confounded 
with the student's generalized attitude of satisfaction with the 
entire college experience* 

What learning measures are available? Most tests are 
inadequate to answer questions of what knowledge is being gained 
by cohorts of students because they were designed for different 
purposes. Tests administered in individual c 1 ass rooms are almost 
invariably course specific; students who did not take those 
courses are at a disadvantage in answering the questions. 



Nationally normed tests in various subject areas are somewhat 
better but they suffer the defect of having been built for 
purposes of screening students. To the traditional test maker, 
the perfect hundred-item test is one that, when it is administered 
to any number of students, yields a mean, median, and modal score 
of 50, with no student missing all the answers and no student 
getting all the items correct. By definition then, half the 
students are below average and half are above. But the test has 
not measured the learning attained by the entire group. 
Nonetheless, those types of tests are popularly employed because 
they are useful in making decisions about program admission and 
course- level placement . 

Some nationally normed tests show more promise. The 
American College Testing Program's College Outcomes Measures 
Project was devised to gain an estimate of student knowledge in 
general education. Several college and university systems are 
begining to use it with student cohorts (Jaschik f 1985). The 
National Assessment of Educational Progress draws samples of 
students aged 9, 13, and 17, showing knowledge for those cohorts 
on specific questions of mathematics and social science, along 
with a smaller set of items in other areas of the liberal arts. 
However, NAEP is concerned with age and regional cohorts; they 
have not applied their testing program to individual shcools. 

The main problems in testing community college student 
knowledge are in item selection and population sampling. Of all 
the universe of knowledge, which questions shall be asked when we 
want to discover what our students know? Should the items be 
course specific or should they deal with areas of general 



inquiry? A set oi: items can tap only a minuscule portion of what 
colleges teach or what students should learn. 

Which students shall respond to the test? It is certainly 
unnecessary to test all the students in the college if we want 
only to know what cohorts of students know as compared to other 
cohorts; population sampling can be readily undertaken. 
The General Academic Assessment 

It is feasible to assess student knowledge in general areas 
such as the liberal arts by building an instrument for that 
purpose and by designing the procedures to fit the realities of 
community college student attendance. The Center for the Study 
of Community Colleges did just that recently in cooperation with 
the faculty and staff in four large urban community college 
districts: Chicago, Miami-Dade, Los Angeles, and St. Louis* The 
results of this administration are instructive in what they 
reveal about student knowledge at various stages in the community 
college programs and as this knowledge relates to student 
demographics. The project also demonstrated a procedure that can 
readily employed in administering ::sts of this type. 

The Center had been tracing trends in the liberal arts and 
transfer education in community colleges by surveying students 
and faculty and by tabulating information on curriculum and 
instructional practices. In conjunction with an Andrew W. Mellon 
Foundation-sponsored study of the liberal arts, the Center staff 
developed, field tested and administered a student survey and 
content test that would reveal student knowledge in general 
education and the liberal arts. "This General Academic Assessment 
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(GAA) was designed to assess knowledge among cohorts of students, 
not the learning of individual students. It was prepared so that 
comparisons could be made between students entering the colleges, 
those who had completed one or more semesters, and those who had 
various aspirations. The items were selected so that a student f s 
general knowledge could be assessed regardless of where or when 
that knowledge was gained. 

The (GAA) had representative items in the humanities, 
sciences, social sciences, mathematics, and English usage. The 
survey portion of the instrument asked students such background 
questions as age, number of credits earned, educational and 
occupational aspirations, number of liberal arts courses taken, 
and self-assessment of their skills in those areas. Items for 
the content portion were drawn from several sources, The 
National Assessment of Educational Progress provided numerous 
usable items in science, social science, and mathematics. 
Educational Testing Service made items in the humanities and 
social science available on loan for purposes of the project. 
The City Colleges of Chicago provided items in English usage, 
and Miami-Dade Community College provided items in the 
humanities. Center staff members culled the items with the help 
of instructors, counselors, and administrators from several 
California community colleges. Final selection was made by 
panels of faculty and administrators from the colleges in which 
the instrument was to be given. 

The items were arrayed in five forms and tried out with 
around 1,300 students in five community colleges in California 
and Kansas. Questions in the demographic portion of the 
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instrument were selected from those that had proved useful in the 
Center's earlier surveys of more than 12,000 students in 
Washington and California. After revision, the final instrument 
included 19 demographic and student experience questions and 57 
items in humanities, 60 in English usage, 52 in mathemathics, 59 
in science, and 71 in social science. The items were distributed 
at random across the five forms so that each form had a sampling 
of items in the five areas and each could be completed within one 
50 minute class period. 

Class sections were used as the unit of sampling because 
that is the most feasible way of getting a random sample of 
students enrolled in credit classes. This method has the 
disadvantage of skewing the sample in the direction of full-time 
students because a student taking four classes has four times as 
many chances of being in a sampled class section as a student who 
is taking only one class. Nevertheless, the sample, based on 
duplicated head count, is an accurate representation since the 
full-time students represent higher proportions of the full-time 
equivalent enrollment in the college. 

A total of 8,026 students in the 23 participating colleges 
(9 in Los Angeles, h in Miami, 3 in St. Louis, and 7 in Chicago) 
completed the form. Their responses were tallied according Co 
total score and to individual sub-tests in humanities, sciences, 
social sciences, mathematics, and English usage. Scores were 
converted to 10 point scale scores for each of the areas and a 
cumulative scale score was tallied. 

Of the students in the sample, 41% were aged 20 or less and 
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another 41% were between age 21 and 30* Twenty-nine percent of 
them had completed between 0-14 units, 22% between 15-29 units, 
19% between 30-44 units, 14% between 45-59 units, and 17% 
indicated that they had completed 60 or more units. Their 
primary reason for attending college: prepare for transfer, 
57%; gain skills necessary to enter a specific occupation, 27%; 
gain skills necessary to advance in a current occupation, 9%; 
satisfy a personal interest, 7%. The sample included 10% Asians, 
26% blacks, 42% whites, 18% Hispanics, Twenty-three percent said 
English was not their native language. 
Results 

The findings were illustrative of the diversity among 
community college students, however they suggest that student 
knowledge is related both to age and to the number of courses 
that students have completed in particular areas. The age 
relationships showed up primarily in English usage, social 
sciences, and humanities; the older the student, the higher the 
score on those scales. These age-related differences did not 
occur in mathematics or science. All of the five scales showed a 
direct relationship between the number of units a student had 
completed and the scores made* The highest scores were made by 
students attending college for their personal interest, with those 
preparing for transfer making the second highest scores on the 
combined scales. And scores tended to increase for each of the 
scales to the extent that students took courses in those areas, 
with the greatest difference being in the humanities and the 
least difference in English usage as related to number of courses 
completed in English composition. 
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One of the more interesting findings was that the students 1 
own assessment of their skills in each of the academic areas 
showed the highest correlation with their score on that scale. 
As an example, students we re asked, "Compared with other students 
at this college, how would you rate your ability to understand 
the implications of scientific and technological developments?"* 
Those students who rated themselves as "poor" scored 4.44 on'the 
science scale while those who rated themselves as "excellent" 
scored 6.34 on that scale. Students who rated themselves as 
"poor" in their ability to "edit written material" scored 4.16 on 
the Fnglish usage scale while those who rated themselves as 
"excellent" scored 6.28. Similar findings obtained for the 
scales in mathematics when students were asked to "use algebra to 
solve problems" (3.82 vs. 6.17), for the scale in humanities 
when they were asked to rate their ability to "understand art, 
classical music, and drama," (2.66 vs 4.22), and for social 
science when they were asked to rate their ability to "understand 
different political ideologies" (4.22 vs 5.82). 
Conclusion 

The administration of a General Academic Assessment 
demonstrated the feasibility of a form of cohort testing that is 
rarely employed in community colleges. The instrument and 
procedure cannot be used to assign students to classes or to make 
any other decision about individuals. However, it can be useful 
for estimating the differences in knowledge exhibited by entering 
students as compared with the cohorts that have completed a year 
or two of coursework at the institution. Furthermore, it can be 
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used to assess entering or graduating students' abilities from 
year to year, thus gaining a measure of overall value added in a 
time series. 

There was no surprise in the finding that students who take 
more courses in an area are likely to know more about that area 
than students who take fewer courses. The age-related scale 
scores proved interesting in that mathematics and science 
knowledge showed little age-related differences; those areas seem 
to be school-related whereas the humanities, social science, and 
English usage abilities may be enhanced merely by living in and 
interacting with the culture. One of the more surprising 
findings was the accuracy of the students 1 self assessment. The 
extremely high correlation between students' self-rating of their 
abilities and their score made on the scale in that area suggests 
that community college students are quite realistic, at least in 
regard to their academic prowess. Students know what they know. 

Center staff engaged in the project as a demonstration of 
what could be done in assessing student knowledge of the liberal 
arts. The study methodology proved effective in gaining a 
measure of student ability. Items were selected by community 
college staff members for their content in each of the five 
liberal arts areas, with particular attention that the items not 
be course specific; accordingly, the instrument proved a valid 
test of general education, within the limits of a quick-score, 
multiple-choice test. 

Because many of the items were borrowed from the copyright 
holders for use in "this one-time research project, the General 
Academic Assessment will not be made available for use in other 
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community colleges. However, the design of the entire project i 
available and might well be adopted for use in other 
institutions. The procedure and the results are detailed in 
Riley (1984), available from the ERIC Document Reproduction 
Service . 
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